{
 "cells": [
  {
   "cell_type": "markdown",
   "id": "c148b65e-e8a6-476e-86ba-bf6a73d479c7",
   "metadata": {},
   "source": [
    "# RAG over the Caltrain Weekend Schedule \n",
    "\n",
    "<a href=\"https://colab.research.google.com/github/run-llama/llama_cloud_services/blob/main/examples/parse/caltrain/caltrain_text_mode.ipynb\" target=\"_parent\"><img src=\"https://colab.research.google.com/assets/colab-badge.svg\" alt=\"Open In Colab\"/></a>\n",
    "\n",
    "This example shows off LlamaParse parsing capabilities to build a functioning query pipeline over the Caltrain weekend schedule, a big timetable containing all trains northbound and southbound and their stops in various cities.\n",
    "\n",
    "Status:\n",
    "| Last Executed | Version | State      |\n",
    "|---------------|---------|------------|\n",
    "| Aug-19-2025   | 0.6.61  | Maintained |"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "ef115dbe-b834-4639-828e-e2c11aef710b",
   "metadata": {},
   "source": [
    "## Setup\n",
    "\n",
    "Download the data."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "335ce1d0-757a-4f09-846e-21c409768871",
   "metadata": {},
   "outputs": [],
   "source": [
    "!wget \"https://www.caltrain.com/media/31602/download?inline?inline\" -O caltrain_schedule_weekend.pdf"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "45fa9120-65bb-4772-9db7-53e7cecf9adc",
   "metadata": {},
   "source": [
    "## Initialize LlamaParse\n",
    "\n",
    "Parse the text results from `LlamaParse`, which will represent complex documents incl. text, tables, and figures as nicely formatted text."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "54aa9579-84d4-49bc-ab54-5474e69c1188",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Started parsing the file under job_id d162724f-dcb9-4bfe-9bd4-337244906fb8\n",
      ".."
     ]
    }
   ],
   "source": [
    "from llama_cloud_services import LlamaParse\n",
    "\n",
    "result = await LlamaParse(\n",
    "    parse_mode=\"parse_page_with_agent\",\n",
    "    model=\"openai-gpt-4-1-mini\",\n",
    "    high_res_ocr=True,\n",
    "    adaptive_long_table=True,\n",
    "    outlined_table_extraction=True,\n",
    "    output_tables_as_HTML=True,\n",
    "    api_key=\"llx-...\",\n",
    ").aparse(\"./caltrain_schedule_weekend.pdf\")\n",
    "\n",
    "documents = result.get_text_documents(split_by_page=True)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "602756b2-9ea1-4519-a8e3-c773ec624205",
   "metadata": {},
   "source": [
    "Take a look at the below text (and zoom out from the browser to really get the effect!). You'll see that the entire table is nicely laid out."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "4928281a-591a-4653-b451-b2b8112a7101",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "          Printer Friendly WEEKEND Caltrain Schedule\n",
      "                                          Morning to Early Afternoon Page 1 of 2\n",
      "          Northbound – WEEKEND SERVICE to SAN FRANCISCO                                                                                                         6XX Local\n",
      "          Train No.             601  603           605  607          609  611            613       615     617       619          621  623            625  627           629  631\n",
      "                 Tamien         6:51a             7:51a             8:51a                9:51a               10:51a              11:51a              12:51p             1:51p\n",
      "       San Jose Diridon         6:56a    7:26a    7:56a    8:26a    8:56a     9:26a      9:56a     10:26a    10:56a    11:26a    11:56a    12:26p    12:56p    1:26p    1:56p    2:26p\n",
      "          Santa Clara           7:03a    7:33a    8:03a    8:33a    9:03a     9:33a      10:03a    10:33a    11:03a    11:33a    12:03p    12:33p    1:03p     1:33p    2:03p    2:33p\n",
      "ZONE 4         Lawrence         7:08a    7:38a    8:08a    8:38a    9:08a     9:38a      10:08a    10:38a    11:08a    11:38a    12:08p    12:38p    1:08p     1:38p    2:08p    2:38p\n",
      "\n",
      "          Sunnyvale             7:12a    7:42a    8:12a    8:42a    9:12a     9:42a      10:12a    10:42a    11:12a    11:42a    12:12p    12:42p    1:12p     1:42p    2:12p    2:42p\n",
      "               Mountain View    7:16a    7:46a    8:16a    8:46a    9:16a     9:46a      10:16a    10:46a    11:16a    11:46a    12:16p    12:46p    1:16p     1:46p    2:16p    2:46p\n",
      "          San Antonio           7:19a    7:49a    8:19a    8:49a    9:19a     9:49a      10:19a    10:49a    11:19a    11:49a    12:19p    12:49p    1:19p     1:49p    2:19p    2:49p\n",
      "              California Ave    7:22a    7:52a    8:22a    8:52a    9:22a     9:52a      10:22a    10:52a    11:22a    11:52a    12:22p    12:52p    1:22p     1:52p    2:22p    2:52p\n",
      "              Palo Alto         7:25a    7:55a    8:25a    8:55a    9:25a     9:55a      10:25a    10:55a    11:25a    11:55a    12:25p    12:55p    1:25p     1:55p    2:25p    2:55p\n",
      "ZONE 3    Menlo Park            7:27a    7:57a    8:27a    8:57a    9:27a     9:57a      10:27a    10:57a    11:27a    11:57a    12:27p    12:57p    1:27p     1:57p    2:27p    2:57p\n",
      "\n",
      "           Redwood City         7:32a    8:02a    8:32a    9:02a    9:32a     10:02a     10:32a    11:02a    11:32a    12:02p    12:32p    1:02p     1:32p     2:02p    2:32p    3:02p\n",
      "             San Carlos         7:35a    8:05a    8:35a    9:05a    9:35a     10:05a     10:35a    11:05a    11:35a    12:05p    12:35p    1:05p     1:35p     2:05p    2:35p    3:05p\n",
      "                     Belmont    7:38a    8:08a    8:38a    9:08a    9:38a     10:08a     10:38a    11:08a    11:38a    12:08p    12:38p    1:08p     1:38p     2:08p    2:38p    3:08p\n",
      "              Hillsdale         7:41a    8:11a    8:41a    9:11a    9:41a     10:11a     10:41a    11:11a    11:41a    12:11p    12:41p    1:11p     1:41p     2:11p    2:41p    3:11p\n",
      "           Hayward Park         7:43a    8:13a    8:43a    9:13a    9:43a     10:13a     10:43a    11:13a    11:43a    12:13p    12:43p    1:13p     1:43p     2:13p    2:43p    3:13p\n",
      "                   San Mateo    7:46a    8:16a    8:46a    9:16a    9:46a     10:16a     10:46a    11:16a    11:46a    12:16p    12:46p    1:16p     1:46p     2:16p    2:46p    3:16p\n",
      "          Burlingame            7:48a    8:18a    8:48a    9:18a    9:48a     10:18a     10:48a    11:18a    11:48a    12:18p    12:48p    1:18p     1:48p     2:18p    2:48p    3:18p\n",
      "                    Broadway    7:51a    8:21a    8:51a    9:21a    9:51a     10:21a     10:51a    11:21a    11:51a    12:21p    12:51p    1:21p     1:51p     2:21p    2:51p    3:21p\n",
      "ZONE 2    Millbrae              7:54a    8:24a    8:54a    9:24a    9:54a     10:24a     10:54a    11:24a    11:54a    12:24p    12:54p    1:24p     1:54p     2:24p    2:54p    3:24p\n",
      "\n",
      "                   San Bruno    7:57a    8:27a    8:57a    9:27a    9:57a     10:27a     10:57a    11:27a    11:57a    12:27p    12:57p    1:27p     1:57p     2:27p    2:57p    3:27p\n",
      "          S. San Francisco      8:00a    8:30a    9:00a    9:30a    10:00a    10:30a     11:00a    11:30a    12:00p    12:30p    1:00p     1:30p     2:00p     2:30p    3:00p    3:30p\n",
      "          Bayshore              8:05a    8:35a    9:05a    9:35a    10:05a    10:35a     11:05a    11:35a    12:05p    12:35p    1:05p     1:35p     2:05p     2:35p    3:05p    3:35p\n",
      "          22ⁿᵈ Street           8:10a    8:40a    9:10a    9:40a    10:10a    10:40a     11:10a    11:40a    12:10p  12:40p      1:10p     1:40p     2:10p     2:40p    3:10p    3:40p\n",
      "ZONE 1    San Francisco         8:15a    8:45a    9:15a    9:45a    10:15a    10:45a     11:15a    11:45a    12:15p  12:45p      1:15p     1:45p     2:15p     2:45p    3:15p    3:45p\n",
      "\n",
      "EFFECTIVE September 21, 2024                                 Timetable subject to change without notice                        See Page 2 For Afternoon and Evening Times\n"
     ]
    }
   ],
   "source": [
    "print(documents[0].text)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "8f5064d4-3e33-4f67-9b2e-46787161538f",
   "metadata": {},
   "source": [
    "## Initialize Query Engine\n",
    "\n",
    "We now initialize a query engine over this data. Here we use a baseline summary index, which doesn't do vector indexing/chunking and instead dumps the entire text into the prompt."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "b3e985b6-9d38-449f-9cf9-aae166824eed",
   "metadata": {},
   "outputs": [],
   "source": [
    "from llama_index.core import SummaryIndex\n",
    "from llama_index.llms.openai import OpenAI\n",
    "\n",
    "llm = OpenAI(model=\"gpt-5-mini\", api_key=\"sk-...\")\n",
    "index = SummaryIndex.from_documents(documents)\n",
    "query_engine = index.as_query_engine(llm=llm)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "66eb0976-2cd6-4b14-9083-124baae9ed5d",
   "metadata": {},
   "outputs": [],
   "source": [
    "response = query_engine.query(\n",
    "    \"What are the stops (and times) for train no 609 northbound?\"\n",
    ")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "7dc6f275-07f4-429e-9335-f50982fe974c",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Train No. 609 northbound (stops and times):\n",
      "\n",
      "- Tamien — 8:51a\n",
      "- San Jose Diridon — 8:56a\n",
      "- Santa Clara — 9:03a\n",
      "- Lawrence — 9:08a\n",
      "- Sunnyvale — 9:12a\n",
      "- Mountain View — 9:16a\n",
      "- San Antonio — 9:19a\n",
      "- California Ave — 9:22a\n",
      "- Palo Alto — 9:25a\n",
      "- Menlo Park — 9:27a\n",
      "- Redwood City — 9:32a\n",
      "- San Carlos — 9:35a\n",
      "- Belmont — 9:38a\n",
      "- Hillsdale — 9:41a\n",
      "- Hayward Park — 9:43a\n",
      "- San Mateo — 9:46a\n",
      "- Burlingame — 9:48a\n",
      "- Broadway — 9:51a\n",
      "- Millbrae — 9:54a\n",
      "- San Bruno — 9:57a\n",
      "- S. San Francisco — 10:00a\n",
      "- Bayshore — 10:05a\n",
      "- 22nd Street — 10:10a\n",
      "- San Francisco — 10:15a\n"
     ]
    }
   ],
   "source": [
    "print(str(response))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "229c4cb0-cf94-4a9f-bc7c-590388f50c1f",
   "metadata": {},
   "outputs": [],
   "source": [
    "response = query_engine.query(\n",
    "    \"What are all the trains (and times) that end at Redwood City going Southbound?\"\n",
    ")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "51cf03ff-7728-4815-ab72-3bf54fc4a2c0",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "None. On this weekend schedule no southbound trains terminate at Redwood City — every listed southbound train continues beyond Redwood City to later stations (Menlo Park/Palo Alto and onward).\n"
     ]
    }
   ],
   "source": [
    "print(str(response))"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": ".venv",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 5
}
